Evaluation of multimodal dialog systems

This paper presents the results of an elaborate study on pen and speech-based multimodal interaction systems. The performance o f the “COM IC” system is assessed through human factors analyses and evaluation o f the acquired multimodal data. The latter requires tools that are able to m onitor user input, system feedback, and performance of the multimodal system components. Such tools can bridge the gap between observational data and the complex process o f the design and evaluation of multimodal systems. The evaluation tool presented here is validated in a human factors study on the usability o f COMIC for design applications and can be used for semi­ automatic transcription of multimodal data.